Rotationally invariant 3D shape contexts using asymmetry patterns

This paper presents an approach to resolve the azimuth ambiguity of 3D Shape Contexts (3DSC) based on asymmetry patterns. We show that it is possible to provide rotational invariance to 3DSC at the expense of a marginal increase in computational load, outperforming previous algorithms dealing with the azimuth ambiguity. We build on a recently presented measure of approximate rotational symmetry in 2D defined as the overlapping area between a shape and rotated versions of itself to extract asymmetry patterns from a 3DSC in a variety of ways, depending on the spatial relationships that need to be highlighted or disabled. Thus, we define Asymmetry Patterns Shape Contexts (APSC) from a subset of the possible spatial relations present in the spherical grid of 3DSC; hence they can be thought of as a family of descriptors that depend on the subset that is selected. This provides great flexibility to derive different descriptors. We show that choosing the appropriate spatial patterns can considerably reduce the errors obtained with 3DSC when targeting specific types of points

Compensating inaccurate annotations to train 3D facial landmark localisation models

In this paper we investigate the impact of inconsistency in manual annotations when they are used to train automatic models for 3D facial landmark localization. We start by showing that it is possible to objectively measure the consistency of annotations in a database, provided that it contains replicates (i.e. repeated scans from the same person). Applying such measure to the widely used FRGC database we find that manual annotations currently available are suboptimal and can strongly impair the accuracy of automatic models learnt therefrom. To address this issue, we present a simple algorithm to automatically correct a set of annotations and show that it can help to significantly improve the accuracy of the models in terms of landmark localization errors. This improvement is observed even when errors are measured with respect to the original (not corrected) annotations. However, we also show that if errors are computed against an alternative set of manual annotations with higher consistency, the accuracy of the models constructed using the corrections from the presented algorithm tends to converge to the one achieved by building the models on the alternative,more consistent set

Craniofacial landmark localisation with asymmetry patterns shape objects

We present a new family of 3D geometry descriptors based on the asymmetry patterns present in the popular 3D Shape Contexts (3DSC)[1]. Our approach resolves the azimuth ambiguity of 3DSC, thus providing rotational invariance, at the expense of a marginal increase in computational load, outperforming previous algorithms dealing with the azimuth ambiguity. We build on a recently presented measure of approximate rotational symmetry in 2D [2] defined as the overlapping area between a shape and rotated versions of itself, to extract asymmetry patterns from a 3DSC in a variety of ways, depending on the spatial relationships that need to be highlighted or disabled. Thus, we define Asymmetry Patterns Shape Contexts (APSC) [3] from a subset of the possible spatial relations present in the spherical grid of 3DSC; hence they can be thought of as a family of descriptors that depend on the subset that is selected. This provides great flexibility to derive different descriptors. We quantify the performance of the proposed descriptors for craniofacial landmark localization, targeting 22 points relevant in the context of dysmorphology research [4]. Measuring the performance in terms of distance to expert annotations we show that APSC can achieve overall accuracy comparable to 3DSC; the rotational invariance of APSC is achieved at the expense of a small computation overhead to build the descriptor (typically < 10%) but implies a speedup during matching by a factor of twice the number of azimuth bins (typically 24 : 1). Moreover, the possibility to define APSC descriptors by selecting diverse spatial patterns from a 3DSC has two important advantages: 1) choosing the appropriate spatial patterns can considerably reduce the errors obtained with 3DSC when targeting specific types of points; 2) Once one APSC descriptor is built, additional ones can be built with only incremental cost. Therefore, it is possible to use a pool of APSC descriptors to maximize accuracy without a large increase in computational cost

Comparing 3D descriptors for local search of craniofacial landmarks

This paper presents a comparison of local descriptors for a set of 26 craniofacial landmarks annotated on 144 scans acquired in the context of clinical research. We focus on the accuracy of the different descriptors on a per-landmark basis when constrained to a local search. For most descriptors, we find that the curves of expected error against the search radius have a plateau that can be used to characterize their performance, both in terms of accuracy and maximum usable range for the local search. Six histograms-based descriptors were evaluated: three describing distances and three describing orientations. No descriptor dominated over the rest and the best accuracy per landmark was strongly distributed among 3 of the 6 algorithms evaluated. Ordering the descriptors by average error (over all landmarks) did not coincide with the ordering by most frequently selected, indicating that a comparison of descriptors based on their global behavior might be misleading when targeting facial landmarks

3D facial landmark localization using combinatorial search and shape regression

This paper presents a method for the automatic detection of facial landmarks. The algorithm receives a set of 3D candidate points for each landmark (e.g. from a feature detector) and performs combinatorial search constrained by a deformable shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing so that the probability of the deformable model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, substantially reducing the number of possible combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in a set of 144 facial scans acquired by means of a hand-held laser scanner in the context of clinical craniofacial dysmorphology research. Using spin images to describe the geometry and targeting 11 facial landmarks, we obtain an average error below 3 mm, which compares favorably with other state of the art approaches based on geometric descriptors

A quantitative assessment of 3D facial key point localization fitting 2D shape models to curvature information

This work addresses the localization of 11 prominent facial landmarks in 3D by fitting state of the art shape models to 2D data. Quantitative results are provided for 34 scans at high resolution (texture maps of 10 M-pixels) in terms of accuracy (with respect to manual measurements) and precision (repeatability on different images from the same individual). We obtain an average accuracy of approximately 3 mm, and median repeatability of inter-landmark distances typically below 2 mm, which are values comparable to current algorithms on automatic localization of facial landmarks. We also show that, in our experiments, the replacement of texture information by curvature features produced little change in performance, which is an important finding as it suggests the applicability of the method to any type of 3D data

Look-alike humans identified by facial recognition algorithms show genetic similarities

The human face is one of the most visible features of our unique identity as individuals. Interestingly, monozygotic twins share almost identical facial traits and the same DNA sequence but could exhibit differences in other biometrical parameters. The expansion of the world wide web and the possibility to exchange pictures of humans across the planet has increased the number of people identified online as virtual twins or doubles that are not family related. Herein, we have characterized in detail a set of “look-alike” humans, defined by facial recognition algorithms, for their multiomics landscape. We report that these individuals share similar genotypes and differ in their DNA methylation and microbiome landscape. These results not only provide insights about the genetics that determine our face but also might have implications for the establishment of other human anthropometric properties and even personality characteristics.This work was funded by the governments of Catalonia (2017SGR1080) and Spain (RTI2018-094049-B-I00, SAF2014-55000, and TIN2017-90124-P) and the Cellex Foundation

The Multiscenario Multienvironment BioSecure Multimodal Database (BMDB)

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1) over the Internet, 2) in an office environment with desktop PC, and 3) in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008Comment: Published at IEEE Transactions on Pattern Analysis and Machine Intelligence journa

The Multiscenario Multienvironment BioSecure Multimodal Database (BMDB)

A new multimodal biometric database designed and acquired within the framework of the European BioSecure Network of Excellence is presented. It is comprised of more than 600 individuals acquired simultaneously in three scenarios: 1 over the Internet, 2 in an office environment with desktop PC, and 3 in indoor/outdoor environments with mobile portable hardware. The three scenarios include a common part of audio/video data. Also, signature and fingerprint data have been acquired both with desktop PC and mobile portable hardware. Additionally, hand and iris data were acquired in the second scenario using desktop PC. Acquisition has been conducted by 11 European institutions. Additional features of the BioSecure Multimodal Database (BMDB) are: two acquisition sessions, several sensors in certain modalities, balanced gender and age distributions, multimodal realistic scenarios with simple and quick tasks per modality, cross-European diversity, availability of demographic data, and compatibility with other multimodal databases. The novel acquisition conditions of the BMDB allow us to perform new challenging research and evaluation of either monomodal or multimodal biometric systems, as in the recent BioSecure Multimodal Evaluation campaign. A description of this campaign including baseline results of individual modalities from the new database is also given. The database is expected to be available for research purposes through the BioSecure Association during 2008